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Abstract 



In this paper, we describe a low-rank matrix completion method based 
on matrix decomposition. An incomplete matrix is decomposed into sub- 
^^rfH matrices which are filled with a proposed trimming step and then are re- 

I ^ combined to form a low-rank completed matrix. The divide-and-conquer 

^— ( approach can significantly reduce computation complexity and storage re- 

fH quirement. Moreover, the proposed decomposition method can be natu- 

rally incorporated into any existing matrix completion methods to attain 
further gain. Unlike most existing approaches, the proposed method is 
^ not based on norm minimization nor SVD decomposition. This makes it 

' ' possible to be applied beyond real domain and can be used in arbitrary 

, fields including finite fields. 

> 

^ 1 Introduction 

Consider a large matrix with only a small portion of known entry, an interesting 
\^ problem is to fill the missing entry assuming the matrix has low-rank. This 

problem has several interesting applications including the so-called collaborative 
filtering problem pp. An example is the famous Nctflix challenge where a huge 
matrix is used to represent the rating of a movie given by a user. Of course, a 
^ typical user will only rate very few movie titles. Therefore, an algorithm will 

be needed to complete the matrix to predict the ratings of all movies among all 
^ users. 

It has been shown theoretically that under certain assumptions the matrix 
can be recovered with very high accuracy [2 [31 0] . Their approaches convert 
the rank minimization problem into a nuclear norm minimization problem in- 
stead and thus can be solved using semidefinite program (SDP). However, the 
complexity still grows rather rapidly with the size of the matrix n ('^ n'^). Sev- 
eral efficient algorithms have been proposed including Singular Value Thresh- 
olding (SVT) [S], Atomic Decomposition for Minimum Rank Approximation 
(ADMiRA) [6], Fixed Point Continuation with Approximate (FPCA) [2, Ac- 
celerated Proximal Gradient (APG) |3, Subspace Evolution and Transfer (SET) 
0, Singular Value Projection (SVP) [TIT, OptSpace [J, and LMaFit HU, where 
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OptSpace and SET are based on Grassmann manifold optimization, SVT and 
SVP uses iterative hard thresholding (IHT) to facilitate matrix shrinkage, FPCA 
utilizes Bregman iterative algorithm and Monte Carlo approximate SVD, and 
LMaFit adopts successive over-relaxation (SOR). 

In this paper, we propose a decomposition method to allow very efhcient 
divide-and-conquer approach when known entries are relatively very few. A 
simple "trimming" method is proposed to recover the decomposed "cluster" 
matrix. However, the decomposition method can also be combine with any other 
existing matrix completion techniques to yield further gain. One advantage of 
the proposed approach is that unlike most existing approaches it does not utilize 
SVD but only relies on basic vector operations. Therefore, the approach is 
immediately applicable to matrices of any field (including finite field matrices). 
This opens up opportunities for new applications. 

The rest of the paper is organized as follows. In the next section, we will fix 
our notation, describe the problem precisely, and present several properties to 
be used in the later sections. Sections |3] and |4] will describe the decomposition 
procedures and present our main results. Section [5] will describe the trimming 
process. 

2 Minimum Rank of Incomplete Matrix 

Let us start with a few notes on our notation. When things are clear, lines of 
partition in matrices will not be shown; the ? sign may represent an unknown 
entry, a row or column of unknown entry, a matrix of unknown entry, etc; and 
similar for the sign. 

Given a finite size matrix M over field F to be completed, let 

S{M) = {M\M is a completion of M}. (2.1) 

If M is already completed, then S{M) = {M}. We define 

mr{M) = min rankM. (2-2) 

MeS(M) 

Such minimum exists because rank(S'(M)) C N and hence 3M G S{M) such 
that 

rankM = mrM. (2.3) 



If M = 



A B 
C D 



3A e S{A) such that mr(Af) ^mr[^, ^ ) , (2.4) 



as we can always find A from M in (2.3). We list other properties about mr{M) 
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that will be quoted: 

mr{M) < rankM, WM G S{M) 

mr{M) < mr{P) if P is any partial completion of M 

mr{[A\B]) < nirA + mrB 

mrM* — mrM 



mr 



A B 
C D 



> mr{A) 



(2.5) 
(2.6) 
(2.7) 
(2.8) 

(2.9) 



mrM — mrN if N can be obtained from M through interchanging of columns/rows. 

(2.10) 

2.1 Junk Row and Junk Column 

Definition 2.1. A row(column) contains entirely either zero or unknown will 
be refered as a junk row(column). 

Certainly, we have 

mr( J) = if J is a junk row (column), (2-11) 

since we can always complete J entirely by zero entries. 

Theorem 2.1. Let M = [J\N] where J is a junk column, then mrM = mrN . 

Proof. By (2.7), we have mrM < mrJ + mrN = mrN. On the other hand 
mrM > mrN by (2.9 1. Hence mrM = mrN. □ 



Thanks to (|2.8|), we have the following corollary: 
= mrN if J is a junk row. 



Corollary 1. mr 



2.2 Equivalence 

We say M is equivalent to TV and write AI ^ N iS N can be obtained from M 
through row interchanging, column interchanging, and junk rows and columns 



deletion and augmentation. By Theorem 2.1 and and (2.10), we have 

M ^ N ^ mrM = mrN. 



(2.12) 



3 Unknown-diagonalization 

Define 



u-diag(Bi,B2,--- ,B„) 



? B2 



7 
7 

Bn 



(3.1) 



We say M is u-diagnonalizable iff M ^ u-diag(^, i?), and both A and B contain 
at least one nonzero known entry. 
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Theorem 3.1. Let M ^ u-diag(Bi, • • • , Bn), then mrM — maa;i<i<„mr(i?i). 



Proof. By (2.12 1 and induction, all we need to show is when M = u-diag(A, B). 
Let mrA = a and mrB = b, by (2.9) we have 

mrM > max(a, b) (3-2) 

Let A, B be completions of A and B, respectively such that rank^ = a and 



(3.3) 



ranks = b (c.f. (|2J)), then by pl3| , 

mrM < mr (u-diag(A, 5)) 



Combining (3.2 1 and ( |3.3[ ), we conclude that 

max{a, b) < mrM < mr (u-diag(A, i?)) 



(3.4) 



By (2.10), we can simply assume the first a columns of A form a basis of 
ColA ; and the first b columns of B form a basis of ColB. Without loss of 
generality, let us assume a> b. We complete the matrix u-diag(^, B) by filling 
up the columns: 



'a; 




'a; 


? 




A 


" ? 




'a; 


A. 




A 


'a; 




'a; 


? 








for 1 < i < 6, 
for 6 + 1 < i < a. 



(3.5) 
(3.6) 



For i > a, we make use of the fact that Ai is a linear combination of {A^. |/c < a}. 
We fill 



Similarly, 



'a; 






A, 
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El 


-I bi,kAk 


A. 






B^ 



for i > a 



for i > b. 



(3.7) 



(3.8) 



A2 
B2 



Now we have a completed u-diag(A, B) and the first a of its columns 

form a basis for its column space. Hence it has 



A, 
Bi 



Ab 
Bb 



At+i 




Aa 





rank a. By (2.5), mr (u-diag(A, i?)) < a and hence by (3.4) and max{a,b) — a, 
we get mrM = a = mr (u-diag(A, B)) as wanted. □ 
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Remark 3.1. Suppose A has been completed by A and a — rank(A). Then 
if the number of column of B = n < a, then we can complete B arbitrarily 



and do the completion in (3.5)-(3.8) as if 6 = n. More generally, given M = 
u-diag(^, B) with A is completed. Then the completing process of B can be 
stopped once we know that the final rank_B will not be greater than rankA no 
matter how we do the remaining completion on B. For example, if size(i?) < 
rank A, where 

size(-B) ~ min(number of column of B, number of row of B), (3-9) 
then we can complete B arbitrarily to start with. 



3.1 Percolation and Clusters 

We would call those Bi,---Bn in Theorem |2.1| as clusters. In other words, 
clusters are matrices that cannot be u-diagonized. They are not the clusters 
in the 2-d square lattice, where each point, not counting the edgy one, has 4 
neighbors. In our case, each entry in an n x to matrix has n + m — 2 neighbors, 
from the view of percolation. Despite that difference, the two models share 
the same percolation threshold at p ~ 0.6 [T2], where p is the occupation rate. 
That means if about 60% of our entries are known, then there is probably one 
cluster left and the matrix cannot be u-diagonalized. We estimate the number of 
clusters as the size of the matrix and the number of known entries vary through 
Monte Carlo simulation and the results are shown in Figure |3.1| We can see 
the number of clusters increases as as the number of known entries increases 
and is peak when the occupation ratio is at about 0.7 regardness the size of the 
matrix. 



3.2 A Decomposition Algorithm 

First of all, we set all junk rows and junk columns of the given matrix to zero 
and blackout them. Now we are working with a junk-free matrix. 

We create a row set and a column set for the first cluster. Then put the row 
position of the first row into the row set; the columns positions of the columns 



with known entries in that row into the column set. Thanks to (2.10), sorting 
is not necessary. We black out the row against repeating searches. For each 
new comers of the column set, we search vertically for its known entries and 
put the corresponding row position into the row sets. After that we black out 
the searched columns. Now the row set may have new comers. We enlarge the 
column set in the way that we enlarged the row set. Both sets keep growing 
until there is no more new comer. 

Then we create another column set and another row set. Repeat the pro- 
cedure for the next cluster, if the remaining matrix has not been blackout to 
void. 
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Figure 3.1: The number of clusters versus k/n, where k is the number of known 
entries and the size of the matrix is n x n. 
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4 Sub Unknown-diagonalization 

Definition 4.1. Let m be a vector and A be a matrix. We define zero{u,A) 
as 1 if V[u|i] € S{[u\A]) (c.f. with rank[u|A] = mr[u\A], then u = 0. 

Otherwise, we define zero{u, A) as 0. 



E.g. zero 



0. Notice that the necessary 



condition (not sufficient) for zero{u, A) = 1 is that u must be a junk column. 

1, if u is junk, 



Hence we have zero{u, A) < junk{u), where junk{u) — 



0, otherwise. 



Theorem 4.1. Let M = 



are vectors that 



zero{vi, Bi)). 



\VnJ 



fvi Bl 

V2 B2 

\Vn ? 



, where Bi are matrices, Vi 



BnJ 



is not a junk column, then mrM ~ maxi(mr[tii|i?i] 



Proof. We wiU show the case when n = 2; cases of higher n are easy to general- 



ized. Let M ~ 
then we have 



A 



B 



with 



is not a junk column. If zero{u, A) = 0, 



mrM > mr[u\A\ 

= mr[u\A\ + zero{u, A), 



(4.1) 



column. Since 



entry A. Let M' 



where the first inequality is by (2.9 1. If zero{u, A) — 1, then u must be a junk 



is not a junk column, v must contains a nonzero known 



u A 
A ?•••? 



By (2.71 and (2.9 1 



(4.2) 



1 + mrA > mrM' > mrA. 

So there are only two possibilies for mrM' . Assume mrM' — mrA. By ( |2.3| 
we can pick [u\A] G S{[u\A]) such that 



mrA — mrM' = mr 



u A 
A ?•••? 



> mr[u|A]. 



(4.3) 



Therefore, mrA — mr\u A\ a s rank[uA\ > rankA > mrA. Since zero{u, A) ~ 1, 
we have u — (c.f. def 4.1 1. But that will make mrM — rankA + 1 because 
A 7^ 0. We conclude that mrM' cannot equal to mrA and hence we must have 



mrM > mrM — mrA + 1 

= mr[M|y4] + zero{u, A). 



(c.f. (|4J) 
(c.f. Theorem lO (4.4) 
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Combine (4.1 ) and (4.4) and the symmetry between {u, A) and {v, B) we get 
mrM > max{mr[u\A] + zero{u, A), mr[v\B] + zero{v, B)). (4.5) 
Without loss of generahty, we may assume 

a = mr[u\A] + zero{u, A) > mr[v\B] + zero{v, B) — b. (4-6) 
Pick [Ao\A] e S{[u\A]),[Bo\B] e S{[v\B]) s.t. 
rarifc[ylo|A] = mr 



and 



rank[Bo\B] ^ mr[v\B]. (c.f. ([2^) 
if zero{u, A) = 0, 



Bq ^ 0, if zero{v, B) = 0. (c.f. def[4T| 

Thanks to ( 2.10[ ), we can assume 

Span{A^, Ai, ....,Aa-i} = Col[Ao\A] 

Span{Bo, B^, = Col[Bo\B] 
Then we complete the rest by filling up the columns: 



for 1 <i <b, 
for b < i < a. 



(4.7) 



(4.8) 



'a; 






? 






" ? 






A. 






'a; 




'a; 


? 








(4.9) 
(4.10) 
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For i > a, we can pick {ai k} G F s.t. Ai = X]o<fe<a '^i,kAk and fill the columns 

(4.11) 

Similarly, for i > 6, we pick {5^,^} G F s.t. Bi = J2o<k<b^'i',k^k and fill the 
columns 

Then the completed matrix is rank a with the first a columns form a basis of 



its column space. By (2.5), 
mrM < a 



= rank[AoA] + zero{u, A) (c.f. (4.6), (4.7)) 

= max(mr[Mj4] + zero{u, A), mr[vB] + zero{v, B)) 
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as assumed. Together with (4.5 1, we get mrM = max(mr[uA]+zero(u, A), mr[vB]~ 
zero(v, B)). 

□ 

Remark 4.1. Suppose [u\A] has been completed by [u\A\ and we have a = 
zero{u, A) + rank[uA]. Then if the number of column of [v\B] = n < a, we 
can complete [v\B] arbitrarily and do the completion in ( |4.9[ )-(4.12 ) as if = n. 
More generally, the completing process of B can be stopped once we know that 
the final zero{v, B) + rank[vB\ won't be greater than a no matter how we do 
the remaining completion on [vB]. For example if junk{v) + size{[vB]) < a (c.f. 
(3.91, def 4.1 1, then we can complete [v\B] arbitarily at the beginning and do 
the completion (|4!9l)-(|4.12|). 



4.1 How to decompose sub u-diagonalizable matrix 

Definition 4.2. A matrix C, not u-diagonalizable, becomes u-diagonalizable 
after deleting a row or a column is called sub u-diagonalizable. The row (column) 
is called conjoined row (column). 



For example. 



V2 



in Theorem 



u-diagonalizable without it. 



4.1 



is a conjoined column and M becomes 



Definition 4.3. Given two vectors v and w of same length, we say w is a donor 
for w {v w) iff all of the unknown positions of v are also unknown in w. In 

fr d\ . _ 

order words, after some row interchanging [wlv] ~ { ^ ] with r and d are 

\? nj 

completed. Clearly, v w and w ^ u imply v > u. However, ii v > w and 
w ^ we do not have v = w. Vectors v and u are said to be comparable if 
either v w or w v. 

Theorem 4.2. Conjoined row (column) does not have donors among other rows 
of the sub u-diag matrix. 

Proof. Let C he the sub u-diag matrix, then it must have the following structure 

'\ 

(after some row and column interchanging) : C = ? B \ , where u cannot be 

\u V J 

entirely unknown, otherwise C is u-diagonalizable. Now, rows in [7\B] cannot 
be donors of [u\v], the conjoined row. Similarly v cannot be entirely unknown 
and hence, rows in [A\7] cannot be donors of [u\v] neither. □ 

Therefore if C is a sub unknown-diagonalizable, we will not miss the chance 
of decomposing it if we have tested every row and column that does not have 
a donor. That is to blackout the suspicious row (column) and then carrying 
out the decomposition mentioned in Section |3.2| We would like to call the 
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decomposed components as sub-clusters. For example, ^^b- 

clusters of the C in the proposition. 

Unhke cluster that cannot be further unknown-diagonalized, sub-clusters 

/A 1 ?\ 





u 


V 


? 




can be sub unknown-diagonalizable. For example C = 


? 


B 


? 


, where 




? 


X 


y 






V 


? 







both [itji;!?] and are conjoined rows. In that case, we may first decompose 

(v ?\ 

. Then we may further decompose the 



C into sub-clusters ( ^ ) and 

u 



B ? 
X y 
V? DJ 

later into | B \ and f , if necessary. 



5 Trimming 

Lemma 5.1. mr[v\M] = mrM if^M e S{M), Col{M) n ^ (/>,. 

Proof. By (1.9) we already have mr[ti|M] > mrM. Let M e S'(Af) such that 



rankM = mrM (c.f. ([2^). Then_pick a w € S'(u) n Col{M). From ([2^, we 
get mr[t;|M] < rank[v\M] = rankM = mrM . □ 

Theorem 5.1. Let Md^ he the di-th column of and donor (c.f. def \4..^ of a 

vector V for 1 < i < t, .such that after some row interchanging, [v\Md-^Md2 ■ ■ ■ Md^.] 

f\D\ _ - _ - 

with r and D are completed. If r E Col{D), then mr[v\M] — mrM . 



1\N 



Proof. Thanks to (2.10), we may start with [v\M] 



Pick Oi e F such that 



f\DA 
1\NB 



(5.1) 



Then VM = ^ S{M), we complete v to 



e S{v)riCol{M). 



(5.2) 



(Note that rank[v\M] = rankM.) Now Lemma 5.1 implies TOr[w|M] = mrM. 

□ 
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5.1 Trimming Process 

We test column by column to see if we can make use of Theorem [O] to trim away 
some columns from a given matrix, which is probably a sub-cluster mentioned 
in the previous section. We call this process as column trimming. When we 
find a column satisfying the condition of Theorem |5.1[ w e will mark down the 



dependency relation between it and its donor (i.e. (5.1)) in order. Then we 
black it out and go for the next column. 



Similarly, we have row trimming. An uninterrupted (c.f. Remarks 3.1 and 



4.1 1 trimming process starts with a column trimming followed by a row trim- 
ming, or the other way round. Then we carry out these two kinds of trimming 
one after the other, until there is no more reduction in the matrix. After the 
trimmed matrix gets completed, we restore, in reverse order, the blackouts with 



the completed forms given by (5.2 1 



The following proposition is interesting in its own right and may be useful 
for our future study. 

Lemma 5.2. If v is a column that S{v) n Col{M) = e S{M). Then 

mr[v\M] — mrM + 1. 

Proof. By (|2j]) mr[v\M] < mrM + I. Let us complete [v\M] by [v\M] such 



that rank[v\M] — mr[v\M] (c.f. (2.2)). Since v ^ Col{M), we must have 



rank[v\M] = rankM + 1, which implies mr[u|Af] — rankM + 1 > rnrM + 1 (by 



(2.5)). □ 



Proposition. Suppose M is a matrix that every two columns Mi and Mj of M 



are comparable (c.f. def 4-3). Then one round of column trimming followed by 
arbitrarily completion and proper restoration (i.e. ( |5.2[ ) ) of the trimmed columns 
complete M to its minimum rank. 

Proof. Let T be the trimmed M. Every two columns of T are also columns of 
M and hence comparable. So there exists a column Tj in T such that Tj ^ Ti 
for all i. After some row interchanging and column interchanging, we have 

Tj = and T — with f and D are completed. Notice that f ^ 

Col{D), otherwise Tj has been trimmed away already. Now Lemma 5.2 



D 



and 



(2.7) give mrN = 1 + mr ( ) • Repeating the argument, we get mrN 



number of columns of iV > rank{N), VN G S{N). Together with Theorem 



5.1 



and (2.5 ), we conclude that mrM = mrN = rank{N), V7V e S{N). Finally, the 



proper restoration (c.f. (5.2)) restores N to M G S{M) such that rank{M) 



rank{N) = mr{M). □ 

5.2 Trimming Process with Approximation 

The trimming process stops when there are no more columns or rows fulfill 



the condition of Theorem 5.1 But we can always make an approximation by 



blacking out a column or a row as if it fulfill the condition and continue the 
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trimming. With this approximation, the process stops when there is no more 
unknown left in the trimming matrix. (We may even choose the row or column 
that has no donor (c.f. Thm 4.2 ) to black out and check for u-diagonalization. 
) 

Then we restore the blackouts in reverse order. When we meet a blackout 
without dependency relation (i.e. (5.1|) to restore, we check for the condition 
of Theorem |5.1[ again. The first time was with the uncompleted trimming 
matrix; this time is with the completed restoring matrix. If the condition is 
fulfilled, we restore the blackout with completed form given by (5.2). This will 
not compromise (further) the minimum rank that we can reach. Otherwise we 
restore the blackout with arbitrary completed form, which may cause one (more) 
rank deviation from the possible minimum. 
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